Field of the Invention
The present invention relates to an image processing apparatus and a method of calculating information according to the motion of a frame.
Description of the Related Art
Along with improvement of the calculation performance of computers, utility of image processing technologies such as region division and alignment in a field called computer vision is growing.
For image alignment, a plurality of motion vectors are calculated from images that are temporally continuous, and motion parameters that express the position shift (motion of the entire image) of the images are calculated from the motion vectors. As the motion vector calculation method, motion search by block matching, a method of detecting feature points, calculating the correspondence between them, and defining the coordinates of corresponding point pairs as motion vectors, or the like is usable (Japanese Patent Laid-Open No. 2007-334625).
The motion parameters can be expressed using two-dimensional vectors, homography matrices, rotation matrices (see Toru Tamaki, “Pose Estimation and Rotation Matrices”, IEICE Technical Report SIP2009-48, SIS2009-23(2009-09)), or the like. However, not all detected motion vectors are always correct, and wrong motion vectors may be included. For this reason, a robust estimation technique of estimating a model from data including errors is necessary.
A representative robust estimation algorithm is RANSAC (see “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography”, M. A. Fischler and R. C. Bolles, Communications of the ACM, 24(6): 381-395, 1981). RANSAC is a technique of estimating an optimum model while repeating calculations. However, the larger the error amount of data is or the larger the number of elements of parameters to be estimated is, the larger the number of repetitions (to be referred to as iterations hereinafter) needed by the RANSAC is. Wrong data will be referred to as “outlier,” and correct data as “inlier” hereinafter.
The image alignment techniques are variously applicable to image blur correction (digital image-stabilization), image composition, encoding, free viewpoint generation, and the like. For example, for a plurality of continuous frames, inverse matrices of matrices expressing the motion between the frames are created by the above-described method. The plurality of inverse matrices are smoothed, and the images are geometrically transformed using the smoothed matrices, thereby correcting motion blurs. The matrix smoothing can be calculated using the moving geometric mean of the matrices. The power root of the matrices necessary to calculate the geometric mean can be calculated using a method described in, for example, “Algorithms for the matrix pth root”, Dario A. Bini, Nicholas J. Higham, and Beatrice Meini, Numerical Algorithms (2005) 39: 349-378.
When calculating, as motion parameters, matrices that express the motion of an entire image from motion vectors including outliers, robust estimation such as RANSAC is necessary. If the outlier ratio of data is high, the RANSAC needs many iterations and takes a long time to process.